Optical pointing devices are already known in the art. U.S. Pat. No. 5,288,993, which is incorporated herein by reference, for instance discloses a cursor pointing device utilizing a photodetector array and an illuminated target ball having randomly distributed speckles. U.S. Pat. No. 5,703,356 (related to the above-mentioned U.S. Pat. No. 5,288,993), which is also incorporated herein by reference, further discloses (in reference to FIGS. 23A and 23B of this document) an optical pointing device in the form of a mouse which does not require a ball and wherein light is reflected directly from the surface over which the pointing device is moved.
The imaging technique used in above-cited U.S. Pat. Nos. 5,288,993 and 5,703,356 in order to extract motion-related information is based on a so-called “Edge Motion Detection” technique. This “Edge Motion Detection” technique essentially consists in a determination of the motion of edges (i.e. a difference between the intensity of pairs of pixels) in the image detected by the photodetector array. Edges are defined as spatial intensity differences between two pixels of the photodetector array. The relative motion of each of these edges is tracked and measured so as to determine an overall displacement measurement which is representative of the relative movement between the photodetector array and the illuminated portion of the surface.
More particularly, according to U.S. Pat. No. 5,288,993, edges are determined between pairs of pixels aligned along a first axis of the photodetector array (for example in each column of the photodetector array) and between pairs of pixels aligned along a second axis of the photodetector array (for example in each row of the photodetector array). FIG. 1 depicts three pixels of a photodetector array, a first pixel or current pixel P, a second pixel Pright aligned with the first pixel P along a first axis 101, and a third pixel Pup aligned with the first pixel P along a second axis 102. Pixels Pright and Pup are shown as being disposed on the right side and top side of pixel P for the purpose of explanation only. It will be appreciated that axes 101 and 102 may be orthogonal (as shown) or alternatively non orthogonal. It will also be appreciated that the pixels are not necessarily disposed so as to form a regular array having rows and columns. Other suitable arrangements may very well be envisaged.
For the purpose of simplification, the pixels of FIG. 1 are either shown as being white or hatched, a hatched pixel denoting an illuminated pixel. In this case, pixel P is illuminated and first and second edge conditions Ex, Ey exist respectively between pixels P and Pright along the first axis 101 and between pixels P and Pup along the second axis 102.
According to U.S. Pat. No. 5,288,993 and U.S. Pat. No. 5,703,356, the displacement measurement is evaluated, on the one hand, based on a normalized difference between the number of edges Ex which move in a first direction along the first axis 101 and edges Ex which move in the opposite direction along the first axis 101 (for example edges which move from left to right and right to left in each row of the photodetector array), and, on the other hand, based on a normalized difference between the number of edges Ey which move in a first direction along the second axis 102 and edges Ey which move in the opposite direction along the second axis 102 (for example edges which move downwards and upwards in each column of the photodetector array).
Relative motion of edges is determined by comparing the position of these edges in the photodetector array at a first point in time with the position of edges in the photodetector array at a subsequent point in time. The optical pointing device thus typically comprises a light source (such as an infrared LED) which intermittently illuminates the portion of the surface in accordance with a determined sequence, and the pixel outputs of the photodetector array are sampled in accordance with the determined sequence to provide basis for the determination of two successive edge frames that are compared to each other in order to determine a relative motion measurement.
According to one embodiment of U.S. Pat. No. 5,288,993 and U.S. Pat. No. 5,703,356 a differential technique is advantageously used in order to determine an edge condition between two pixels. According to this embodiment, an edge is defined as existing between two pixels if the ratio of intensities of the two photosensitive elements is larger than a determined level. An edge may thus be defined mathematically by the following Boolean expression:Intensity[PIXEL 1]>K Intensity[PIXEL 2]ORK Intensity[PIXEL 1]<Intensity[PIXEL 2]  (1)where K is a selected scaling factor.
The edge condition Ex between two pixels along the first axis 101 in FIG. 1 (or horizontal edge condition) may accordingly be defined by the following Boolean expression:Ex=Intensity[pixelP]>KIntensity[pixelPright]ORK Intensity[pixel P]<Intensity[pixel Pright]  (2)Ex at a high state meaning that there exists an edge between the current pixel and the pixel on its right.
Similarly, the edge condition Ey between two pixels along the second axis 102 in FIG. 1 (or vertical edge condition) may accordingly be defined by the following Boolean expression.Ey=Intensity[pixel P]>K Intensity[pixel Pup]ORK Intensity[pixel P]<Intensity[pixel Pup]  (3)Ey at a high state meaning that there exists an edge between the current pixel and the pixel on top.
It will be appreciated that conditions Ex and Ey above are not dependent on the direction or sense of the edge but simply indicate whether an edge exists or not between two pixels.
The above definitions and associated motion detection algorithms described in U.S. Pat. No. 5,288,993 and No. 5,703,356 have been used with great success in optical trackballs, i.e. optical pointing devices which detect light intensity patterns of an illuminated ball which is manipulated by the user. The above definitions and associated algorithms however require that the detected light intensity patterns exhibit clearly defined light intensity differences. Accordingly, the surface of the ball is covered with a plurality of randomly shaped markings in a color which exhibits great contrast with respect to the background. In addition, these markings (generally in the shape of spots) need to have a predetermined size. Referring to U.S. Pat. No. 5,288,993, the size of these markings is for instance said to be in the range of 0.5 mm2 to 0.7 mm2, with a density of about one spot per square millimeter. The size of these markings is relatively independent of the diameter of the ball but however depends largely on the resolution and size of the photodetector array. As mentioned in U.S. Pat. No. 5,703,356, the image of a single spot on the sensor should, as a minimum, cover the center to center distance between two adjacent pixels (or pixel pitch). In practice, however, the preferred typical dot size has been selected so that the surface of the image of the dot covers about five pixels. By complying with the above requirements, a number of edges which remains substantially constant and which is substantially less than the number of pixels may be determined and used for motion detection.
A serious problem however arises when applying the above motion detection scheme to optical pointing devices where the illuminated surface does not exhibit a predetermined pattern, such as in the case of an optical “ball-less” mouse which directly uses light reflected from a random surface, such as paper or the surface of a desk. An optical mouse not requiring any ball is described in U.S. Pat. No. 5,703,356, but this embodiment requires that the illuminated surface exhibits a suitable pattern having sufficient numbers of dark and light areas of sufficient size.
In any case, when applying the above technique to optical sensing devices which use an uncharacterised surface as reference for motion detection, edges as defined above are actually detected between every pixels. Since specific and well-defined patterns of edges cannot clearly be detected and tracked, it is accordingly impossible to derive any measurement of the relative motion between the optical sensing device and the illuminated surface. In practice, the above-mentioned motion detection technique is therefore not applicable and alternative solutions need to be found.
It is accordingly an object of the present invention to provide a solution that is more suited to an application in an optical pointing device where the illuminated surface which is used as reference for motion detection is uncharacterised and produces purely random light intensity patterns.
It should be mentioned at this point that some solutions to the above goal are already known in the art. U.S. Pat. No. 5,578,813 and No. 5,644,139 for instance disclose a motion detection principle which is based on correlation of consecutive image frames outputted by the photodetector array. According to this motion detection principle, comparison of light intensity between neighbouring pixels is however not performed to provide any information concerning spatial intensity differences (i.e. edges) between neighbouring pixels. Correlation of successive image frames as proposed in the just mentioned documents however implies some limitations. In particular, in order to derive a sufficiently precise measurement of relative motion, it is in practice required to illuminate the reference surface with a very low angle with respect to the surface, which angle is typically within sixteen degrees of the surface as more precisely described in U.S. Pat. No. 5,686,720. This leads to constraints in the construction of the optical pointing device.
In addition, image frame correlation techniques as proposed in these documents show some limitations when analysing grayscale image frames, the complexity of the correlation process increasing exponentially with the depth of intensity of the image. In practice, the application of the above correlation techniques to motion detection in optical device is limited to the analysis of binary black-and-white images.
In contrast, detecting spatial intensity differences (i.e. edges) between pixels of the image frame (or in other words pre-processing the image frames to extract binary data descriptive of light intensity differences between neighbouring pixels) and comparing this extracted data with previously extracted data in order to determine a measurement of relative motion is particularly efficient in terms of sensitivity to slight variations of intensity between pixels. With this technique, one can easily extract characterising features and patterns that may be tracked and exploited directly for motion detection. It may also be demonstrated that this technique is particularly robust and simple in terms of required processing.